Skip to content

chore: colocate tree-sitter-requirements.wasm to resolve issues when bundled with webpack#410

Merged
Strum355 merged 1 commit into
guacsec:mainfrom
Strum355:nsc/requirements-txt-treesitter-4
Mar 12, 2026
Merged

chore: colocate tree-sitter-requirements.wasm to resolve issues when bundled with webpack#410
Strum355 merged 1 commit into
guacsec:mainfrom
Strum355:nsc/requirements-txt-treesitter-4

Conversation

@Strum355

Copy link
Copy Markdown
Member

Description

Fourth times the charm? This time I definitely didnt forget to test with vsce

Checklist

  • I have followed this repository's contributing guidelines.
  • I will adhere to the project's code of conduct.

@Strum355
Strum355 requested a review from ruromero March 12, 2026 12:00
@qodo-code-review

Copy link
Copy Markdown

Review Summary by Qodo

Colocate tree-sitter-requirements.wasm for webpack compatibility

✨ Enhancement

Grey Divider

Walkthroughs

Description
• Colocate tree-sitter-requirements.wasm file to resolve webpack bundling issues
• Change from file path resolution to URL-based approach for WASM loading
• Add build scripts to copy WASM file to source and distribution directories
• Load WASM as binary bytes instead of file path reference
Diagram
flowchart LR
  A["tree-sitter-requirements.wasm<br/>in node_modules"] -->|"pretest script"| B["Copy to src/providers/"]
  A -->|"postcompile script"| C["Copy to dist/src/providers/"]
  D["requirements_parser.js<br/>uses URL-based loading"] -->|"reads WASM bytes"| B
  D -->|"at runtime"| C
Loading

Grey Divider

File Changes

1. src/providers/requirements_parser.js ✨ Enhancement +4/-3

Switch to URL-based WASM loading with binary bytes

• Replace fileURLToPath with readFile from node:fs/promises
• Change WASM path resolution from import.meta.resolve() to relative URL using import.meta.url
• Load WASM as binary Uint8Array bytes instead of file path string
• Pass binary bytes to Language.load() instead of file path

src/providers/requirements_parser.js


2. package.json ⚙️ Configuration changes +3/-1

Add build scripts to copy WASM file

• Add pretest script to copy WASM file from node_modules to src/providers/
• Add postcompile script to copy WASM file from node_modules to dist/src/providers/
• Ensure WASM file is colocated with compiled output for webpack bundling

package.json


Grey Divider

Qodo Logo

@qodo-code-review

qodo-code-review Bot commented Mar 12, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (3) 📘 Rule violations (0) 📎 Requirement gaps (0)

Grey Divider


Action required

1. Tests miss WASM copy 🐞 Bug ✓ Correctness
Description
npm run tests / npm run tests:rep do not trigger the new pretest copy step, so
src/providers/requirements_parser.js will attempt to read ./tree-sitter-requirements.wasm (which
is gitignored) and crash with ENOENT.
Code

package.json[R40-46]

		"test": "c8 npm run tests",
		"tests": "mocha --config .mocharc.json --grep \".*analysis module.*\" --invert",
		"tests:rep": "mocha --reporter-option maxDiffSize=0 --reporter json > unit-tests-result.json",
+		"pretest": "cp node_modules/tree-sitter-requirements/tree-sitter-requirements.wasm src/providers/tree-sitter-requirements.wasm",
		"precompile": "rm -rf dist",
-		"compile": "tsc -p tsconfig.json"
+		"compile": "tsc -p tsconfig.json",
+		"postcompile": "cp node_modules/tree-sitter-requirements/tree-sitter-requirements.wasm dist/src/providers/tree-sitter-requirements.wasm"
Evidence
The WASM file is not present in the repo by default (gitignored) and the parser now unconditionally
reads a colocated WASM from disk. Only the test script triggers pretest; running
tests/tests:rep directly (as documented) bypasses that copy, leaving the WASM missing at
runtime.

package.json[37-47]
src/providers/requirements_parser.js[5-10]
.gitignore[1-12]
CONTRIBUTING.md[12-20]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`npm run tests` and `npm run tests:rep` bypass the `pretest` lifecycle hook, but `requirements_parser` now requires a colocated `tree-sitter-requirements.wasm` (which is gitignored). This makes documented developer workflows fail with ENOENT.

### Issue Context
- The WASM file is `.gitignore`d under `src/providers/...`.
- Only the `test` script triggers `pretest`; `tests` and `tests:rep` do not.

### Fix Focus Areas
- package.json[37-47]
- src/providers/requirements_parser.js[5-10]
- .gitignore[1-12]
- CONTRIBUTING.md[12-20]

### Suggested direction
Option A (scripts): add `pretests` and `pretests:rep` that perform the same copy as `pretest`.

Option B (runtime fallback): in `requirements_parser.js`, attempt to `readFile(wasmUrl)` and if it fails, fall back to resolving/loading the WASM from the dependency (e.g., using `import.meta.resolve` + `fileURLToPath`) so tests/dev runs don’t depend on generated artifacts.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Windows test script breaks 🐞 Bug ⛯ Reliability
Description
The new pretest uses the POSIX cp command, causing npm test to fail on Windows even though the
codebase contains Windows-specific logic and appears to support win32.
Code

package.json[43]

+		"pretest": "cp node_modules/tree-sitter-requirements/tree-sitter-requirements.wasm src/providers/tree-sitter-requirements.wasm",
Evidence
npm test will now execute pretest, which shells out to cp. On Windows environments without a
POSIX shell, cp is not available, so tests fail before running. The repo contains explicit win32
handling, indicating Windows is a supported runtime scenario.

package.json[40-46]
src/providers/python_controller.js[66-81]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`npm test` now runs a `pretest` that uses `cp`, which is not available on Windows by default, breaking test execution on win32.

### Issue Context
The codebase includes explicit Windows handling (`os.platform() === &#x27;win32&#x27;`), so introducing POSIX-only build steps is inconsistent with apparent platform support.

### Fix Focus Areas
- package.json[40-46]

### Suggested direction
Change the copy commands to be cross-platform, for example:
- `node -e &quot;import {copyFileSync, mkdirSync} from &#x27;node:fs&#x27;; import {dirname} from &#x27;node:path&#x27;; mkdirSync(dirname(&#x27;src/providers/tree-sitter-requirements.wasm&#x27;), {recursive:true}); copyFileSync(&#x27;node_modules/tree-sitter-requirements/tree-sitter-requirements.wasm&#x27;,&#x27;src/providers/tree-sitter-requirements.wasm&#x27;);&quot;`
(and similarly for the `dist/...` copy).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

3. Redundant WASM initialization 🐞 Bug ➹ Performance
Description
requirements_parser.init() re-reads the WASM from disk and loads the language on every
getParser()/get*Query() call; callers frequently invoke multiple getters per operation,
multiplying filesystem I/O and initialization work.
Code

src/providers/requirements_parser.js[R7-10]

async function init() {
	await Parser.init();
-	return await Language.load(wasmPath);
+	const wasmBytes = new Uint8Array(await readFile(wasmUrl));
+	return await Language.load(wasmBytes);
Evidence
The module’s public getter functions each call init(). With the new implementation, init()
includes readFile(wasmUrl) and Language.load(wasmBytes), so repeated getter usage repeatedly
reads and reloads the WASM. Some callers request several getters at once (including via
Promise.all), increasing the duplicated work.

src/providers/requirements_parser.js[7-31]
src/providers/python_pip.js[98-103]
src/providers/python_controller.js[46-56]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`init()` performs `readFile()` + `Language.load()` and is called by every exported getter, leading to repeated disk reads and repeated language loads within a single operation.

### Issue Context
Call sites often request multiple getters (sometimes concurrently), so duplicated initialization is common.

### Fix Focus Areas
- src/providers/requirements_parser.js[5-31]

### Suggested direction
Implement a module-level cache:
- `let languagePromise;`
- In `init()`, if `languagePromise` is undefined, assign it to the async initialization (Parser.init + readFile + Language.load), otherwise await and return the cached promise.
Optionally also cache `Query` objects if they are immutable and safe to reuse.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

ⓘ The new review experience is currently in Beta. Learn more

Grey Divider

Qodo Logo

@Strum355
Strum355 force-pushed the nsc/requirements-txt-treesitter-4 branch from d38c078 to 892de52 Compare March 12, 2026 12:02
Comment thread package.json
Comment thread package.json
@Strum355
Strum355 merged commit 29f6867 into guacsec:main Mar 12, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants